1. Field of the Invention
The invention relates to an apparatus having a neural network and a method of performing a learning procedure in a neural network.
2. Description of the Prior Art
Learning of a neural network is carried out as follows. Learning data indicative of certain known information are input into a neural network constructed by interconnecting nodes which correspond to neurons in a human brain. Then, by successively changing each value of the connection weight of the nodes (hereinafter referred to as "weight"), the information included in the learning data is learned in the weight system.
Generally, when such learning of a neural network is carried out, a random value is provided as an initial value for each weight for changing the value of the weight. This provision makes it possible to handle a set of input learning data having any value (i.e., any information), when learning information included in the learning data.
Conventionally, if a category to which information indicated by learning data belong is unknown, learning of a neural network is carried out by using a self-organization learning (self-learning) procedure such as the self-organized formation of feature maps by Kohonen or Adaptive Resonance Theory by Grossberg. On the other hand, if a category to which information indicated by learning data belong is known, learning of a neural network is carried out by using a teacher-supervised learning procedure such as the error back-propagation (back-propagation) by Rumelhart or learning vector quantization 2 (LVQ2) by Kohonen.
The learning procedure in either case where the category is unknown or known requires a huge number of calculations until the learning is converged, because respective values of the weights are successively changed depending upon the degree of similarity in the contents between the input learning data and each of the weights, or depending upon the degree of difference in the contents between the information indicated by the input learning data and the information indicated by the output data associated with the input learning data. If a state becomes such that the input learning data is very similar to the weight, or that the contents of the information indicated by the input learning data are very similar to the contents of the information indicated by the output data, the learning is judged to be converged. Such a state is called a learned state.
In order to reduce the number of calculations, an alternative method is employed. In this method, a random value is not used as an initial weight value, and a learning data set is previously clustered by the K-means method (which is a conventional statistical clustering method) or by other methods. Then, using a representative vector or average vector of the respective cluster obtained, each weight value is initialized. According to this alternative method, the number of calculations required until the learning is converged can be greatly reduced.
However, because the algorithm of the K-means method used in clustering the learning data set and an algorithm of the calculation used in learning of the neural network are quite distinctive from each other, it is difficult to use the algorithm of the K-means method in the learning of the neural network. Therefore, learning data are first clustered by the K-means method, and a weight value is initialized by using a resulting representative vector, average vector, etc. Then, learning of the neural network is performed. Since the learning of the neural network requires these two steps, there arises a problem that the learning process is very complicated.
Thus, the invention described herein makes possible the objectives of:
(1) providing an apparatus having a neural network in which learning can be readily performed;
(2) providing an apparatus having a neural network in which learning can be performed with a reduced number of calculations;
(3) providing a method of performing a learning procedure in a neural network by which the neural network can be readily brought into a learned state; and
(4) providing a method of performing a learning procedure in a neural network by which the neural network can be brought into a learned state with a reduced number of calculations.